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Abstract 



This paper concerns a method for finding the minimum of a polynomial on a semialgebraic 
set, i.e., a set in R m defined by finitely many polynomial equations and inequalities, using the 
Karush-Kuhn- Tucker (KKT) system and sum of squares (SOS) relaxations. This generalizes 
results in the recent paper [15) . which considers minimizing polynomials on algebraic sets, 
i.e., sets in R m defined by finitely many polynomial equations. Most of the theorems and 
conclusions in 1151 generalize to semialgebraic sets, even in the case where the semialgebraic set 
is not compact. We discuss the method in some special cases, namely, when the semialgebraic 
set is contained in the nonnegative orthant R™ or in box constraints [a, &]„. These constraints 
make the computations more efficient. 

Keywords: polynomials, semialgebraic sets, Karush-Kuhn- Tucker (KKT) system, Sum of 
Squares (SOS). 

1. Introduction 

In this paper, we consider the optimization problem 



where x = \xi ■ ■ ■ x n ] £ R n and f(x),gi(x),hj(x) £ ~R[x] (the ring of real multivariate 
polynomials in a;). Let T be the feasible region, i.e., the subset of R n which satisfies con- 
straints 11.21 — 11.31 : J- is a semialgebraic set. Many optimization problems in practice can 
be formulated as ll.H - ll.3H . Finding the global optimal solutions to ll.lt — 11.31 is an NP- 
hard problem, even if f(x) is quadratic and g%, hj are linear. For instance, the Maximum-Cut 
problem for graphs is of this form, and it is NP-hard (ED- 

Recently, the techniques of sum of squares (SOS) relaxations and moment matrix methods 
have made it possible to find the global optimal solutions to 11.11 - 11.31 by approximating 
nonnegative polynomials with SOS polynomials, which allows the problem to be implemented 
as a semidefinite program. For more details about these methods and their applications, 
see [HI EH EH ESI E3 El E0 IIH- To prove the convergence of these methods, it is often 
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min f(x) 

s.t. Qi(x) =0, i = 1, •• • , s, 
hj(x) > 0, j = !,-•• ,t 



(1.1) 
(1.2) 
(1.3) 
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necessary to assume that the feasible region T is compact or even finite. In 1221 . it is shown 
that SOS relaxations can solve 111.111 - 11.31 globally in finitely many steps in the case where 
{x £ C n : gi(x) — ■■■ — g s (x) = 0} is finite and the ideal (gl (#),•• • ,g s (x)) is radical. 
If we only assume that {x £ C™ : gi (x) — ■ ■ ■ = g 3 (x) — 0} is finite, it is shown in I I 21 
that the moment matrix method can solve (II. Ill - Ill. 31 globally in finitely many steps. Finally, 
if T is compact and the set of polynomials {gt,hi} satisfies an additional assumption (see 
Theorem 12.41 . then arbitrarily close lower bounds for /* can be obtained by SOS relaxations 
or moment matrix methods fill . In this case, the convergence is asymptotic, however little is 
known about the errors in the bounds. 

The above global optimization methods are based on representation theorems from real 
algebraic geometry for polynomials positive and nonnegative on semialgebraic sets. On the 
other hand, the traditional local methods in optimization often follow the first order optimality 
conditions (zero gradient in the unconstrained case or the Karush-Kuhn- Tucker (KKT) system 
in the constrained case). The underlying idea in 1151 and the present paper is to combine 
these two types of methods in order to more efficiently solve 11.11 - 11.31 globally. In |T5], 
SOS relaxations are applied on the gradient ideal I gra d (the ideal generated by all the partial 
derivatives of f(x)) in the unconstrained case, and on the KKT ideal Ikkt in the constrained 
case, where only equality constraints are allowed. When I gra d or Ikkt is radical, which is 
generically true in practice, the method in 1151 can solve the optimization 11.11 - 11.21 globally; 
otherwise, arbitrarily close lower bounds of /* can be obtained. No assumptions about T are 
made, i.e., it need not be finite or even compact. 

The convergence of the method in 1151 assumes that the constraints are algebraic sets. If 
there are any inequality constraints, T is no longer algebraic but only semialgebraic and the 
proof in |15| does not work. The motivation of this paper is to generalize the method in |15| 
to handle semialgebraic constraints. 

The KKT system of problem JTTJ-JOl is 

s t 

F = V/(x) + £ AiVffi(x) - £ UjVhj{x) = 0, (1.4) 

i=l 3=1 

h j {x)>0,v i h j {x) = Q, j = l, (1.5) 
Si (as) =0, i = !,-■■ ,s, (1.6) 

where vectors A = [Ai ■ • • A S ] T and v = lyi ■ ■ ■ vt\ T are called Lagrange multipliers. See 
1161 for some regularity conditions that make the KKT system hold at local or global mini- 
mizers. For an example where the KKT system fails, see Example l4.2l in Section 4. 

Note that we do not require v > in the above; this makes the SOS relaxations simpler 
and does not affect the convergence of the method, since omitting the constraint v > means 
simply that there are more feasible points for 11.41 - 11.61 . including maxima as well as minima. 
But since we minimize over this larger set, we get the same minima. Minimizing over this 
larger set makes our problem easier, because it reduces the number of inequality constraints, 
which as we will see greatly lowers the complexity of our algorithm. 

Let f K KT be the global minimum of f(x) over the KKT system defined by 11.41 - 11.61 . 
Assume the KKT system holds at the global minimizers. Then we claim that f* = Ikkt- 
First, /* < ffCKT follows immediately from the fact that all x in the KKT system are feasible. 
Now let x* be a global minimizer such that f(x*) = /*, then by assumption, there exist 
Lagrange multipliers A* and v* > such that (x* ,X* ,v*) satisfies the above KKT system. 
Thus /* > fpcKT an d hence they are equal. 



2 



Define the KKT ideal Ikkt and its varieties as follows: 

Ikkt = {Fi,- ■ ■ , F n ,gi, ■•• ,g s , vihi,- ■ ■ , v t h t ) , 

Vkkt = {{x, A, v) € C" x C s x C* : v(x,\,v) = 0, Vp e /jcjct}, 

V^kt = {(a:, A, i/) 6 I™ x I ! x I 1 : p(a;, A, i/) = 0, Vp G Ikkt}. 

Here F = [Fi, ■ ■ ■ ,F n ] T is defined in (O). Let 

H = {(ac.A.i/) e 1" x I s x R 1 : hj{x) > 0, j = 1,- • • ,*}. 

The preorder cone Pkkt associated with the KKT system is 



Pkkt = < ^2 < J eh e 1 1 h 6 2 2 ■ ■ ■ hp 
I 9e{o,i}' 



ere are SOS > + Ikkt- 



The linear cone associated with the KKT system is 

a , • • • , o t are SOS V + Ikkt- 



Mkkt = ^ (Tp + ffjhj 

3=1 



Note that Ikkt C Mkkt Q Pkkt Q M.[x, A, i/]. 

In solving SOS programs, we often set an upper bound on the degrees of the involved 
polynomials. Define the truncated KKT ideal 

n s t 

In, kkt = I <f>kFk+ y>g» + i/)jVjhj\deg(<l>kF k ), deg(ipigj), deg(ipjVjhj) < ivj. 

fc=l i=l 3=1 

and truncated preorder and linear cones 



Pn.kkt = < ^ agh 6l h e 2 2 ■ ■ ■ h Bt 



sefo.i}* 



degiaghl 1 ■ ■ ■ h 9t ) < N } + In, i 



J (to, • • • ,a t are SOS 1 

Mn,K KT = |- ' ^/'A. deg(<70 ), deff((T>J ) < iv} 



+ In, KKT- 



A sequence {p/v} of lower bounds of 11,11 - 11.31 can be obtained by the following SOS 
relaxations: 

piJv = max 7 (1.7) 

s.t. f(x) ~j e Pn.kkt- (1.8) 

Since Pn,kkt has a summation over 2* terms like crgh® 1 h 2 2 ■ ■ ■ h 9t , it is usually very expensive 
to solve the SOS program 11.71 - 11.81 in practice. So in practice, it is natural to replace the 
truncated preorder cone Pn,kkt by truncated linear cone Mn,kkt, which leads to the SOS 
relaxations: 

f% = max 7 (1.9) 
s.t. f(x) - 7 e M N ,kkt- (1-10) 
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Thus we have the increasing sequences of lower bounds {/jv}?f=2 an d {Pat}?/=2 such that 

The following notation is used throughout: We denote by deg(p) the degree of a polynomial 
p. The vector inequality u < v (u,v £ R") is defined component-wise, i.e., Uj < Vi for each i. 
[it, v\ n denotes the set of all vectors w £ R n such that u < w < v. 

This paper is organized as follows. Section 2 is a review of some fundamental results from 
algebraic geometry. In Section 3 we discuss the representation of the polynomial f(x) in the 
cones Mkkt and Pkkt- We analyze the convergence of the lower bounds {p* N } and {fly} in 
Section 4. In Section 5, we consider some special cases of inequality constraints, in particular, 
the nonnegative orthant and the box [a, b] n . Section 6 draws conclusions. 

2. Preliminaries 

This section will introduce some basic notions from algebraic geometry needed for our 
discussion. Readers may consult f° r more details. In this section, all polynomials are 

in the indeterminate x = (xi, . . . ,x m ) for the simplicity of notation. Here x is not the "a;" 
in the Introduction, but rather a generic indeterminate. In later sections, x will be again the 
u x" in 11.11 - 11.31 . and all polynomials will be in the variables (x, A, v), unless explicitly stated 
otherwise. 

We write M.[x] = M.[xi, . . . , x m ] for the ring of polynomials in indeterminates x = (asi, . . . , x m ) 
with real coefficients. A polynomial p £ M[x] is SOS if it can be written as a sum of squares 
of polynomials in M[x]. A subset / of M[x] is an ideal if p ■ q <E I for any p £ I and q £ M[x]. 
For pi, . . . ,p r £ WL[x], (pi, ■ ■ ■ ,p r ) denotes the smallest ideal containing the pi. Equivalently, 
(pi, - •■ ,p r ) is the set of all polynomials that are polynomial linear combinations of the pi. 
Every ideal arises in this way: 

Theorem 2.1 (Hilbert Basis Theorem) 

Every ideal I C R[x] has a finite generating set, i.e., I = (pi, ■ • ■ ,pi) for some pi, ■ ■ • ,pi £ I. 

The variety of an ideal I is the set of all common complex zeros of the polynomials in I: 

V(I) = {X £ <C m : p(X) = for all p £ /}. 

The subset of all real points in V(I) is the real variety of I. It is denoted 

V m (I) = {X £ R m : p(X) = for all p £ /}. 

If I = (pi, . . . , Pr ) then V(I) = V( Pl , . . . , Pr ) = {X £ C m : pi(X) = ■ ■ ■ = p r (X) = 0}. An 
ideal I C R[X] is zero-dimensional if its variety V(I) is a finite set. This condition is much 
stronger than requiring that the real variety V R (I) be a finite set. For example, / = (Xi 
is not zero-dimensional, however the real variety V K (I) = {(0, 0)} consists of one point of the 
curve V(I). 

A variety V C C m is irreducible if there do not exist two proper subvarieties V\,V% S V 
such that V = V\ U V^. The reader should note that in this paper, "irreducible" means that 
the set of complex zeros cannot be written as a proper union of subvarieties defined by real 
polynomials. Given a variety V C C m , the set of all polynomials that vanish on V is an ideal 

I(V) = {p £ R[x] : p(u) = for all u £ V}. 

Given any ideal / of M[x], its radical is the ideal 

\Tl = {q€R[X] : q l £ J for some I £ N}. 

Note that / C \fl. We say that J is a radical ideal if yl = I. Clearly, the ideal I(V) defined 
by a variety V is a radical ideal. The following theorems offer a converse to this observation: 
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Theorem 2.2 (Hilbert's Weak Nullstellensatz) 

If I is an ideal in R[as] such that V(I) = then lei. 

Theorem 2.3 (Hilbert's Strong Nullstellensatz) 

If I is an ideal in R[x] then I(V(I)) = \/7. 



Remark. Theorems 12, 21 l2~3l are normally stated for ideals in C[x]. However, keeping in mind 
that V(I) lies in C m , they hold as stated. 

In real algebraic geometry, we are also interested in subsets of R m of the form 

S = { X G R m : pi(X) — ■ ■ ■ — p r (X) — 0, qi(X) > 0, ■ • ■ , qi(X) > 0}, 

where Pi,q 3 € M,[x]. Such S is called a basic closed semialgebraic set. Given S as above, the 
preorder and linear cones associated with S are defined as 



P ( S ) = £ MX)q e 1 1 (X)---q e /(X) 

{ S6{0,l> f 

M(S) = \<T (X) + Y^1j(X)a j (X) 



(To, 



, at are SOS 



Pi, 



■Vr 



3=1 



0-0,0-1, 



at are SOS > + (pi, ■■ ■ ,p r 



A linear cone or preorder M is archimedean if there exists p(x) £ M such that the set 
{I 6 I™ : p(X) > 0} is compact, equivalently, if there exists N G N such that N - J27Li x i £ 
M. Note that if AI(S) or P(S') is archimedean, then S is compact. 

Theorem 2.4 (Putinar, Suppose M(S) is archimedean, then every polynomial p(x) 

which is positive on S belongs to M(S). 



Remark. There are examples of compact S for which M(S) is not archimedean and the 
conclusion of Putinar's Theorem does not hold. In the case of the preorder P(S), it is a deep 
theorem of Schmiidgen 12 II that if S is compact then P(S) is archimedean and any polynomial 
which is positive on S is in P(S). For this reason, the SOS relaxations p* N always converge to 
the minimum if S is compact, however, the relaxations f^ may not converge to the minimum. 
On the other hand, it is sometimes the case in practice that we know or can compute some 
JV £ N such that our semialgebraic set S is contained in the sphere {TV — Tl,T=i x i — 
this case, we can simply add one additional constraint, namely TV — Y^iLi x i — 0> an d force 
M(S) to be archimedean. 

The sets P(S) and M(S) contain the ideal J = (pi, ■ ■ ■ ,p r ). If J is radical and V(J) is 
finite, we have the following theorem: 

Theorem 2.5 (Parrilo, 1221 ) Let S and J be defined as in the above. Suppose J is a zero- 
dimensional radical ideal in R[X]. Then a polynomial w(x) € R[X] is nonnegative on S if and 
only ifw(x) £ M(S). 

For a semialgebraic set, there is a well-known generalization of the Hilbert's Weak Null- 
stellensatz, see e.g. [SJ 4.2.13]. 

Theorem 2.6 Suppose S and P(S) are defined as above, then S = if and only if — 1 £ P(S). 
We need the following lemma from )15| : 
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Lemma 2.7 (Lemma 3.2. I15H Let Vi,-- - ,V r be pairwise disjoint varieties of C m . Then 
there exist polynomials pi, ■ ■ ■ ,p r £ C[X] such that Pi(Vj) = 5ij, where 5ij is the Kronecker 
delta function. 

Furthermore, if each Ve is conjugate symmetric, i.e., a point z £ C m belongs to Ve if and only 
if its complex conjugate z £ Ve, then the polynomials pe can be chosen such that pe £ R[X], 
since we can replace Pi(x) by (pi(X) + pi(X))/2, where pi(X) is obtained from Pi(X) by 
conjugating its coefficients. 

3. Representations in Pkkt and Mkkt 

In |15l . it is shown that if a polynomial f(x) £ R[a;] is globally nonnegative and its gradient 
ideal is radical, then f(x) has a representation as a sum of squares modulo the gradient ideal. 
In this section we generalize this result to real polynomials which are nonnegative on the 
semialgebraic set Vkkt'- We will show that such polynomials have a representation in Pkkt 
modulo the ideal Ikkt, if the later is radical. Furthermore, in some cases we can replace the 
preorder cone Pkkt by the linear cone Mkkt- 

Throughout this section we fix a polynomial f(x) £ M.[x] along with an optimization of the 
form (1.1)-(1.3) and the corresponding ideal Ikkt, variety Vkkt, the preorder cone Pkkt 
and the linear cone Mkkt- 

/From Theorem 12.51 we immediately obtain the following representation theorem: 

Theorem 3.1 Assume Ikkt is zero- dimensional and radical. If f{x) is nonnegative on 
Vkkt n H, then f(x) belongs to Mkkt- 

Using a proof similar to that of Theorem 3.1 in 1151 . we can remove the restrictive hypothesis 
that Ikkt be zero-dimensional, however to obtain the most general result we must replace 
the linear cone Mkkt by the preorder cone Pkkt ■ 

Theorem 3.2 Assume Ikkt is radical. If f(x) is nonnegative on Vkkt H fi, then f(x) 
belongs to Pkkt- 

We need a generalization of a lemma from |15| : 

Lemma 3.3 Let W be an irreducible component of Vkkt- Then f(x) is constant on W. 

Proof. We first note that 

s t 

F(x) =f(x) + Y, A ^( x ) + E v i h M 
i=i j=i 

is equal to f(x) on Vkkt, and the right hand side has zero gradient on Vkkt- With this in 
mind, the proof of 1151 3.3] generalizes easily to this case. □ 

Proof of Theorem VJ.'A Decompose Vkkt into its irreducible components, then by Lemma f3.3l 
f(x) is constant on each of them. Let Wo be the union of all the components whose inter- 
section with TL is empty, and group together the components on which f(x) attains the same 
value, say Wi, . . . , W r . Suppose f(x) = q 4 > on Wi. 

We have Vkkt — Wo U W\ U ••■ U W r , and Wi are pairwise disjoint. Note that by 
our definition of irreducible, each Wi is conjugate symmetric. By Lemma 12.71 there exist 
polynomials po,pi, - ■ ■ , p r £ A, v] such that pi (Wj ) = 8ij , where Sij is the Kronecker delta 
function. 
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By assumption, Wo n Tt = and so, by Theorem 12.61 there are SOS polynomials vg (8 £ 
{0, 1}*) such that 

-1= vehl 1 ■ ■ ■ h / d = v mod I (Wo). 

ee{o,i} 4 

We have / = (/ + i) 2 - (f + (±) 2 ) = /j + v ■ f 2 for the SOS polynomials /i = (/ + |) 2 , /„ = 
/ 2 + (i) 2 . Then 

/ = /i + W0/2 = u ^°\ ■■■f l ° t t d = qo mod I (Wo) 

ee{o,i}* 

for some SOS polynomials ug (8 £ {0, 1}*). Recall that f(x) = oti, a constant, on each 
Wi(l <i<r). Set qi (x) = yfa~, then /(z) = q t (x) 2 on i"(W»). 

Now let q = qo (po) 2 + Y^i=i (liPi) 2 ■ Then f — q vanishes on Vkkt and hence / — q £ Ikkt 
since Ikkt is radical. It follows that / £ Pkkt- □ 

Remark. The assumption that Ikkt is radical is needed in Theorem l3.2l as shown by Exam- 
ple 3.4 in I15| . However, when Ikkt is not radical, the conclusion also holds if f(x) is strictly 
positive on Vkkt- 

Theorem 3.4 If f(x) is strictly positive on Vkkt f^H then f(x) belongs to Pkkt- 

Proof. As in the proof of Theorem l3.2l we decompose Vkkt into subvarieties Wo , Wi , ■ • ■ , W r 
such that Wo PI TC = 0, and for i = 1, . . . r, Wi Pi TL 7^ and / is constant on Wj. Since each 
Wi, i > contains at least one real point and f(x) > on Vkkt-, each a, > 0. The Wi were 
chosen so that each en is distinct, hence the Wi's are pairwise disjoint. 

Consider the primary decomposition Ikkt — ff i= oJi corresponding to our decomposition 
of Vkkt, i-e., V(Ji) = Wi for i = 0, 1, • ■ • , r. Since Wi n Wj = 0, we have Ji + J 3 = R[x, A, u] 
by Theorem 12.21 The Chinese Remainder Theorem, see e.g. |1] 2.13], implies that there is an 
isomorphism 

p : R[x, A, v]/I K kt -> R[cc, A, v\j Jo X R[x, A, i/]/Ji x ■ • ■ x R[x, A, i/]/j r . 

For any p € M[x, A, v], let [p] and p([p])i denote the equivalence classes of p in R[x, A, 1/]/ Ikkt 
and R[a;, A, 1/]/ Ji respectively. 

Recall that that V(Jo) n 7i = 0, hence by Theorem 12.61 there exist SOS polynomials 
u g (9 £ {0, 1}') such that 

-1= ^2 u 9 p([/i* 1 ]) • • • p([/i?'])o < = i ' "0 mod J . 

ee{o,i} f 

As in the proof of Theorem l3.2l we write / = /1 — /a for SOS polynomials /1, /2 and then we 
have 

/s/i + iio/as ^ u »(p(K 1 ]))o • • • (p(K']))o = go mod J 
es{o,i}* 

for some SOS polynomials vg (8 £ {0, 1}'). Thus the preimage p~ 1 ((qo, 0, • • • ,0)) £ Pkkt- 

Now on each Wi, 1 < i < r, f(x) = at > 0, and hence (f(x)/on) — 1 vanishes on Wi. Then 
by Theorem 12.31 there is I £ N such that (f(x)/cti — 1) £ Ji. /.From the binomial theorem, it 
follows that 

(1 + (f(x)/ ai - 1)) 1/2 ee ( ^ ) (/(x)/ai - l) fc ^ qi/y/EH mod Ji . 



7 



Thus (p([f]))i = qf is SOS in R[x, X, v\j Ji, and hence p^ 1 {qfe i+ t) is SOS in R[x, X,u}/ Ikkt, 
where ej+i is the (i + l)-st standard unit vector in R r+1 . 

Finally, we see that p([f]) = (qo, Qi , • • • ,1r)- The preimage of the latter is 

r 

((*), ?i, • • ■ , <7r)) = (?oei)) + ^ {qf e i+ i) , 

i=l 

which implies that / G Pkkt- □ 

Remark. The conclusions in Theorem 13.21 and Theorem 13,41 can not be strengthened to show 
that f(x) £ Mkkt- The following is a counterexample. 

Example 3.5 Consider the optimization 

min f(x) = (x'3 — x\x2) 2 — 1 + e 
s.t. /ii(x) = 1 — x 2 > 
/i2(x) = 2:2 > 
/i3(a:) = X3 — X2 — 1 > 

where < e < 1. ^From the constraints, we can easily observe that the global minimum 
f*—e>0 which is attained at x* = (0, 0, 1). Its KKT ideal 

Ikkt = {2xiX2(x 3 — x\xi) — V\X\,2x\{xz — x\x-x) +1*2 — vz, 



2{x- 3 - x\x%) - 1/3,^1(1 - x\), V2X2, v z (xz - x 2 - 1)^> 



is radical (verified in Macaulay 2 However, we can not find SOS polynomials ero, d, o~i,o~z 
and general polynomials 4>\,4>2,<t ) 3 such that 

df df df 
f(x) = ct + o\h\ + a 2 h 2 + 0-3/13 + <M~, u i x 2) + <M a v ^ + v $) + 03 (7; vz). 

OXl OX2 OXz 

Suppose to the contrary that they exist. Plugging v = (0, 0) in the above identity yields 

= 1 — e + (To + — Xl) + O2X2 + cr 3 (x' 3 — x 2 — 1) + <t>(xz — x\x2) 

where <j> = —Ax\<f>\ — x\<f>2 + 2<f>z — {x'i — x\xy). Now substitute xz = x\x2 in the above, yielding 

a 3 ((l — x\)x2 + 1) = 1 — e + a + iti(1 — x{) + a 2 x 2 . 

Here 00, o~\, 02, o~z are now considered as SOS polynomials in {xi, X2). Since 1 — e > 0, 03 can 
not be the zero polynomial. If 03 = <jz(xi) is independent of X2, we can derive a contradiction 
using an argument identical to the argument in the proof of of \21A Thm. 2]. Thus 2m — 
deg x ^az(x\,X2) > 2 and 2d — deg xi <jz(x\, X2) > 0. On the left hand side, the leading term is 
of the form A ■ x 2d+2 x 2rn+1 with coefficient A < 0. Since the degree in X2 on the left hand 
side is odd, the leading term on the right hand side must come from o~2{xi,X2)x2, and is of 
the form like B ■ x\ d x^ ri+1 with B > 0. This is a contradiction. Therefore we can conclude 
that f(x) Mkkt- 



4. Convergence of the Lower Bounds 

In this section, we will show that the lower bounds {p"pf} obtained from 11.71 - 11.811 converge 
to /* in 11.11 - 11.31 . The conclusions in Section 4 of |15l can generalized, based on Theorem 13. 2 1 
and Theorem 13.41 in the preceding section. However, we need an extra assumption to ensure 
the convergence of {/jv}- 
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Theorem 4.1 Assume f* is finite and the global optimizers x* of fl.lp - fl.3p satisfy the KKT 
system \1.4p - U.bV . Then lim p* N — f* . Furthermore, if Ikkt is radical, then there exists 

some N £ N such that p* N = /*, i.e., the SOS relaxations fl.7p - ll.86 converge infinitely many 
steps. 

Proof. The sequence {p* N } is monotonically increasing, and p* N < f* for all N £ N, since 
/* is attained by f(x) in the KKT system ll.4H - ll.6ll by assumption and the constraint 11.101 
implies that 7 < /*. Now for arbitrary e > 0, let y e = /* — e and replace f(x) by f(x) — 7,, in 
11.11 - 11.31 . The KKT system remains unchanged, and f(x) — j e is strictly positive on V K kt- 
By Theorem 13.41 f(x) — 7 e £ Pkkt- Since f(x) — 7 E is fixed, there must exist some integer 
N\ such that f(x) — 7 e £ Pn u kkt- Hence /* — e < p* Nl < f* . Therefore we have that 
lim p* N = f*. 

N— >oo 

Now assume that Ikkt is radical. Replace f(x) by f(x) — f* in 11.11 - 11.31 . The KKT 
system still remains the same, and f(x) — /* is now nonnegative on Vkkt- By Theorem 13. 21 
f(x) — /* £ Pkkt- So there exists some integer N2 such that f(x) — f* £ Pn 2 ,kkt, and 
hence P^r, > /*. Then p* N < f* for all N implies that p* N ^ = /*. □ 

Remarks. (1) In Lasserre's method jll| . a sequence of lower bounds that converge to /* 
asymptotically can be obtained when the feasible region T is compact; but those lower bounds 
usually do not converge in finitely many steps. However, from Theorem 14.11 we see that when 
Ikkt is radical then the lower bounds {p* N } converge in finitely many steps, even if T is not 
compact. This implies that the lower bounds {ppf} may have better convergence even in the 
case where T is compact. 

(2) The assumption in Theorem H.ll can not be removed, which is illustrated by the following 
example. 

Example 4.2 Consider the optimization: min x s.t. x s > 0. Obviously f* = and the global 
minimizer x* — 0. However, the KKT system 

1 - v ■ 3a; 2 = 0, v ■ x 3 = 0, x :i > 0, v > 

is not satisfied, since Vkkt = 0- Actually we can see that the lower bounds {/at} given by 
ll.y6 - ll.lU6 tend to infinity. By Theorem VZ.'A Vkkt = implies that 1 £ Pkkt, i.e., 

(1 + 3^2, ,2 )(1 — 3^x 2 ) + 9u 2 x ■ ux 3 — 1. 

In the SOS relaxation fl.Up - fl.lUp . for arbitrarily large 7, x — 7 £ Pkkt, since 

x - 7 = (x - 7)(1 + 3^a- 2 )(l - 3^a; 2 ) + 9v 2 x(x - 7) • ux 3 £ Pkkt. 

Thus pg = 00. In this example, the conclusion in Theorem \4-l\ does not hold. 

The convergence of lowers bounds {/jv} cannot be guaranteed, as we see in Example 13.51 
In that example, replace the objective by the perfect square (0:3 — xix 2 ) 2 . Then /* = 1, but we 
do not have lim = 1. ^From the arguments there, we can see that f(x) — (1 — e) ^ Mkkt 

N—*oo 

for all < e < 1, which implies that /Jy < 0. But /iv > is obvious since (X3 — x\x2) 2 is a 

perfect square. Therefore lim /jv = < 1 = /*, i.e., the lower bounds {/aV} obtained from 
jv-^oo 

Hl.90 - Hl.lOp may not converge. 

On the other hand, the situation is often not that bad in practice. In the examples in the 
rest of this paper, it always happens that lim p* N = lim /w = /*. If we further assume that 

Mkkt is archimedean then it must hold that lim p* N = lim = f* from Theorem 12.41 
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(Putinar). This is the generalization of assumption 4.1 in 1 1 1 1 . See also the remark after 
Theorem 1211 

The SOS relaxation itD^- ltntjl can be solved using software SOSTOOLS |5T]- The dual 
problem of 11,91 - 11.101 is to minimize a linear functional over some linear moment matrix 
inequalities. It can also be obtained by applying moment matrix methods to minimize f(x) 
over the semialgebraic set defined by KKT system 11. 41 - 11. 61 . The dual problem can be solved 
using software Gloptipoly |HJ. Actually, the formulations of SOS relaxations and moment 
matrix methods are dual to each other, see 1111 1121 . The SOS relaxations II. 91 - 11.101 not 
only give the lower bounds /jv, but also the information about global minimizers x* and 
their Lagrange multipliers (X*,v*). SOSTOOLS can extract the minimizer if the moment 
matrix has rank one. Gloptipoly can also find the lower bounds, and extract the global 
minimizers when the moment matrix satisfies some rank condition. Gloptipoly does not need 
the moment matrix to be rank one. The tricks to extract global minimizers in Section 5.2 
in II 51 can be applied here directly to find (x* , A*, v*), so omit further discussion. For more 
details about how extracting minimizers from SOS relaxations or moment matrix methods, 
see 0. 

Example 4.3 (Exercise 2.18, 1101 ) Consider the global optimization: 

min (-4xf + xl)(3xi+4x 2 - 12) 
s.t. 3si - 4a; 2 < 12, 2xi - x 2 < 0, -2xi - x 2 > 0. 

The global minimum f* = -18.6182 and the minimizer x* = (-24/55,128/55) « (-0.4364,2.3273). 
The lower bound obtained from H.9t) - ll.l(A) is fX = —18.6182. The extracted minimizer 
x = (-0.4364,2.3273). 

Example 4.4 Consider the Quadratically Constrained Quadratic Program ( QCQP): 

4 2 2 2 „ 
mm - -x L + -x 2 - 2X!X 2 

s.t. x 2 — x\ > 0, — x\x 2 > 0. 

The global minimum /* = and minimizer x* — (0,0). The feasible region T defined by the 
constraints is non-compact. The lower bound returned by \l.yjl - \l.lUti is /| = —2.6 x 10~ 15 
(Note: this computation was done in double precision floating point, with round off error 
bounded by 2~ 53 ~ 10" 16 / The extracted minimizer is x = (6.1 x 10~ 16 ,-9.0 x 10" 17 ) and 
the Lagrange multiplier is v = (0.3884, 0.3909). 



5. Optimization over Some Special Semialgebraic Sets 

In problem H1.9II - I1.1UII . the polynomials are in (x, A, v) £ R n+S +* which means that when 
there are many constraints, the problem is very expensive to solve. If u(x, A, v) is a polynomial 
of degree d, it can have ^ n + a +'+ d ) coefficients; this will be huge for large s, t, or d. Frequently, 
if the polynomials gi(x) and hj{x) are of some special form, then the KKT system 11.41 - 11.61 
can be simplified and hence the SOS relaxations 11.91 - 11.101 will be easier to solve. In this 
section we look at the case where {a; £ R" : h\(x), ■ • • , ht(x) > 0} is the nonnegative orthant 
R" or the box [a,fe] n and show how these type of problems can be simplified. 
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5.1. Minimizing Over the Nonnegative Orthant R™ 

In this subsection, suppose the inequality constraints 11,31 are the standard constraints for 
the nonnegative orthant K™ := {x G R n : x\ > 0, • • • , x n > 0}. The constraints are of the 
form 

= • • ■ = g s {x) = 0, a; e R+. 
Then the KKT system PI-PI becomes 

V/(x) + J)AiVsi(x)-i/ = 0, 

i=l 

gi(x) = ••• = g s (a;) = 0, 
Zfc^fc =0, fc = 1, ••• ,n, 
v € R™. 

In this KKT system, the variable f can be solved for explicitly. By eliminating ^, the above 
system simplifies to 

Ox fe f-f (9x fe 

1—1 

9i(x)=... = fl .(x) = 0. (5.2) 

We define cones M X ^ T and M^ KKT similar to the definition of Mkkt and Mm,kkt (see 

Section 1), define associated to the above simplified system. Note that M I ^ CT ,MJj CT C 
M\x, A] and the Lagrange multiplier v does not appear. Similar to a sequence 

{/at} °f lower bounds of 11.11 - 11. 311 can be obtained by the following SOS relaxations: 

fx = max 7 (5.3) 
s.t. f{x)-j€Ml+ KKT . (5.4) 

Now the indeterminates in the above SOS program are (a;, A) instead of (x,X, v). Thus a 
polynomial u(x, \) of degree d has at most (™ + ^ +d ) coefficients, which is much smaller than 
{ n+ "d t+d ) wnen ~t is large. This makes solving 15.31 - 15.41 much less expensive. 

Since v — (ui, ■ ■ ■ ,v t ) are eliminated by direct substitutions, systems 11.41 - 11.61 and 15.11 - 

l|5.2|l are equivalent. Thus we see that f(x) — j G Mn 1 ,kkt if and only if f(x) — r y £ Mn!> kkt> 
for some integers iVi and Na. Therefore the lower bounds {/iv} have the same property of 
convergence as {fx} obtained from PI - PHI . 

If, in addition, the equality constraints 11.21 are hyperplanes, i.e., the constraints are the 
standard simplex: 

Ax = b, x > 

where A G R sx ", b G R s , then the KKT system PJ - PJ can be reduced to 

df T 
Xk(^r — ha fc A) = 0, k = 1, ••• ,n 
ox k 

Ax = 6, x > 
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where at £ R s is the fc-th column of matrix A 

Furthermore, if Ax = b consists of 
the KKT system has the simpler form 



Furthermore, if Ax = b consists of a single equation a T x = b 7^ 0, then A = — - v ^ x ^ and 



, df x T Vf(x), „ , „ 
a x = b, x > 

where a = [ai,--- , a,i] T . 

Based on the above two simplified KKT systems, SOS relaxations similar to can 
be obtained immediately, improving the computational efficiency. 

Example 5.1 (Test Problem 2.9, H3) Consider the Maximum Clique Problem for n = 5: 

4 

min — ( S~] Xi^i+i + xiXb + X1X4 + x 2 x 5 + X3X5) 

i=l 

s.t. sci + £2 + X3 + X4 + x 5 = 1 

X!,X2,X 3 ,X4,X 5 > 0. 

The global minimum f* = —1/3 and minimizers x* are (1/3, 1/3, 0, 0, 1/3), (1/3,0,0,1/3,1/3), 
(0,1/3,1/3,0,1/3), and (0,0,1/3,1/3,1/3). The lower bound obtained from f5^)-J5^p is 
/I = -0.33333333378814. The difference f - ft w 4.5 x 10~ 10 . 

Example 5.2 (Exercise 1.20, |10p Consider the optimization: 

n—l 

\ ^ 2 2 

mm > XiXi+i + x n x\ 



s.t. 



x-i = 1, k > 0. 



27ie global minimum f* = and t/ie minimizers are the vertices of the simplex defined by the 
constraints. The lower bound obtained from 15.51) - p"7| ) is /J = —4.0 • 10~ 8 . 

Example 5.3 f(x) = x T Hx and the constraints are < x < e, where x G R 5 and e = 
[1,1,1,1,1] T , and 



H = 



1 


-1 


1 


1 


-1 


1 


-1 


1 


1 


-1 


1 


-1 


1 


1 


-1 


1 


-1 


1 


1 


-1 



is a co-positive matrix (MiA i.e., f(x) > Vi > 0. If each x% is replaced by x\, then 

the resulting quartic polynomial is nonnegative, but not SOS. Consider the Quadratic Program 
(QP): 

min x T Hx 

S.t. Xl, X2, X3, £E4, X5 > 0. 

The lower bound obtained from f,5..'j|) - f?^| ) is /| = —3.35 x 10~ 9 . Actually, we have the 
following decomposition 



x T Hx = + 2 • (xi ■ hf 
\5.ty - \57%\l . Here hi is the i-th column of matrix H. 
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5.2. Minimizing Over the Box 

In this subsection, we consider the case that 11.31 are given by box constraints, i.e., x G 
[a, b] n where a = [ai • • • a n ] T and 6 = [6i • ■ • 6„] T . Here we assume that a < b. In 
this case, the feasible region T is compact, and Lasserre's method can be applied here. 
However, as remarked after Theorem 14.11 if Ikkt is radical then our method will converge 
after finitely many steps. Usually Lasserre's method has only asymptotic convergence. 

Now the KKT system iTT^-lPl has the form 

S 

V/(a:) + J2 A * V S' ( X ') - v + V = °> 

gt{x) =■■■ = g s (x) = 0, 
(xk - ak)fk = 0, (bk - Xk)fJ.k = 0, fc = l,---,n, 
x — a > 0, 6 — a; > 0, 

where Vi(/j,i,Xi) is the i-th component of Lagrange multipliers v(fi,X) respectively. One good 
property of this KKT system is that the vectors v and fi can be solved for explicitly. Elimi- 
nating v and (i, we obtain 

^Jaf~ + ^ Jsc^ 3 * ~~ afc ^ &fe _ a ' fe ) = °' k = 1,-- ■ ,n, 

i — l 

gi(x) = • • ■ = g s (x) = 0, x — a > 0, fe — x > 0. 

Like the definition of M^x T and M^ r KKT (see the preceding subsection), define the cones 
M-kkt alm ^-n'kkt associated with the above simplified KKT system, where M^'xt , ^d KKT 
M[x, A]. Similar to l|5.3|l - H5.4|l . a sequence of lower bounds {/at} of l|l.l[l - l|1.3[l can be obtained 
by the following SOS relaxations: 

&=max 7 (5.5) 

s.t. /(i)- 7 £MM» t . (5.6) 

Now a polynomial u(x, A) of degree d in M]^'^^- T has at most (™ + ^ +d ) coefficients, which is 
much smaller than (J l + B +^ n + ^ the number of coefficients of one polynomial of degree d in 
Mn,kkt- So 15. 51 - 115.611 can be solved much more efficiently. Similarly as {/at}, the lower 
bounds {/at} have the same properties of convergence as {/at}. 

Consider the special case that f(x) = \x T Hx + g T x is a quadratic function and there are 
no equality constraints. Here and H = H T G R nxn is symmetric. The the above KKT 

system can be further reduced to 

(hk x + g k ){x k — a k )(bk — Xk) = 0, k = 1, ■ ■ ■ ,n, 
x—a > 0, b — x > 0. 

Here hk(gk) is the fc-th row (component) of arrays H(g). Finding the global minimum of a 
general nonconvex quadratic function over a box is an NP-hard problem. The relaxations 
15.51 - 15.61 provides a new approach for such nonconvex quadratic programming. 
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Example 5.4 (Test Problem 4.7, |SJ) Consider the optimization: 

min — 12xi — 7x2 + x\ 

s.t. - 2x\ + 2 - x 2 = 

< X! < 2, < x 2 < 3. 

27ie best known objective value is —16.73889. The lower bound obtained from 15. 51) - 15, 61) is 
/g = —16.73889. So /* = /g. 27ie extracted minimizer x — (0.7175,1.4698) and Lagrange 
multiplier A = —4.0605. 

Example 5.5 (Test Problem 2.1, Consider the optimization: 

5 

min 42xi + 44x 2 + 45x 3 + 47x 4 + 47.5x 5 - 50 V] X; 
s.t. 20xi + 12x-2 + II13 + 7x 4 + 4x 5 < 40 

< Xi, X2, X3, X4, X 5 < 1. 

The global minimum f* = —17 and the minimizer x* = (1, 1, 0, 1, 0). The lower bound obtained 
form l5~5\)-T5~o\) is / 6 * = -17.00. The extracted minimizer x = (1.00, 1.00, 0.00, 1.00, 0.00) and 
Lagrange multiplier v = 0.1799. 

Example 5.6 (Exercise 2.22, 1101 ) Consider the Maximum Independent Set Problem 

min -J2 X >+ 

XiXj 

s.t. < Xi < 1, i = 1, • • • , n. 

TTie negative of the global minimum — /* equals the cardinality of the maximum independent 
vertex set of G = (V, i5). Let G be a pentagon with two diagonals which do not intersect in the 
interior. Now n = 5 and f = —2. TTie lower bound obtained from 15. 51) - 15. 61) is /| = —2.00. 

Example 5.7 (Exercise 1.32, 1101 ) Consider the optimization: 



mm ]~[ Xi — ^ Xi 

i = l i = l 

s.t. < o < xi, ■ • ■ ,x n < b. 



The global minimum f* — a n — na when a > 1. For n = 4, a = 2, = 3, t/ie lower bound 
obtained from t5~5\) - I^W is / 6 * = 8.00. The extracted minimizer is x = (2.00, 2.00, 2.00, 2.00). 



6. Conclusions 

This paper generalizes most of the theorems in 1151 from optimizations constrained by 
algebraic sets to optimizations constrained by semialgebraic sets, under the assumption that 
the global minimizers satisfy the KKT system. The special structures of the KKT system are 
exploited to accelerate the algorithm when the constraints include the nonnegative orthant 
R" or the standard box [a, b]„. 

In general, the SOS relaxations ll.9H - ll.10ll are very hard to solve when there are many con- 
straints, which introduces many Lagrange multipliers. So the structures of ll.9H - ll.10ll should 
be exploited to improve the efficiency of the method. Section 5 discusses the specifications 
with the nonnegative orthant R" and the standard box [a, b] n . 
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